Many application programs make use of the Internet Protocol (IP) when being accessed across local and wide area networks as well as across the Internet. While the use of IP has advantages, it also exposes these applications as well as corporate networks to certain risks. Therefore, companies must be vigilant in regulating traffic passing into and out of their corporate/enterprise networks, irrespective of origin and with due attention paid to the content, or payload, of data packets in addition to packet header information.
Traditionally, firewalls have been used to regulate enterprise traffic at the IP packet level. First-generation firewalls are essentially packet filters, which act by inspecting individual packets as they pass between different computer systems. If a packet matches one of the packet filter's rules, the packet filter takes the corresponding action prescribed by that rule. Firewall rulesets may be “inclusive” or “exclusive”. An exclusive firewall allows all traffic through except for the traffic matching a rule from the ruleset. An inclusive firewall does the reverse.
Packet filters typically operate at a relatively low level of the TCP/IP protocol stack (typically OSI Layer 2 to Layer 4). The firewall administrator may define the matching criteria for the firewall along with the corresponding rules for how to treat packets upon a match. Packet filters generally do not take action according to whether individual packets are part of existing traffic flows, that is, the packet filters do not maintain any information concerning connection state. Instead, packets are filtered based only on information contained within the individual packets themselves.
Second-generation firewalls, on the other hand, are “stateful” filters, which maintain records of connections passing through the firewall. Any existing network connection can be described by several properties, including source and destination IP address, UDP or TCP ports, and the current stage of the connection's lifetime (including session initiation, handshaking, data transfer, or completion connection). Because stateful firewalls maintain context about active sessions, they can use that state information to speed up packet processing. If a packet does not match an existing connection, it will be evaluated according to the firewall's ruleset for new connections. If a packet matches an existing connection based on comparison with the firewall's state table, it will be processed in accordance with rules for that connection.
An example of a well-known firewall application is ipfw, an open source firewall software implementation. ipfw uses rules and associated coding techniques to retain information about active connections. Rule evaluation in ipfw, however, is not stateful.
The ipfw rule set is organized such that groups of ipfw rules are coded to allow or deny packets based on the values contained in those packets. When a packet enters the firewall it is compared against the first rule in the rule set and such comparisons progress one rule at a time in ascending rule number sequence order. As soon as the subject packet matches a particular rule's selection parameters, that rule's action field value is executed and the search of the rule set terminates for that packet. This is referred to as a “first match wins” search method. If the packet does not match any of the rules, it is acted on by an ipfw default rule, which allows or denies the packet according to the associated action specified by the firewall administrator.
Another form of network device used to regulate traffic in computer networks is a proxy server (often referred to simply as a proxy). Generally, a proxy is a computer system or application program that resides logically between one or more clients and one or more content sources (e.g., servers), and which terminates connections between the clients and the content sources. In response to a client request, for example for a specified service, file, connection, web page, or other resource, the proxy provides the requested content directly (e.g., if it stores a local copy of same) or connects to the specified server and makes the request on behalf of the client. In this latter case, the proxy may retain a copy of the requested content so as to service later requests for that content directly, without having to connect to the server. Transparent proxies are so termed because their presence in a network is transparent to the clients (and possibly also to the servers as well). That is, the client does not need to configure a proxy and cannot directly detect that its requests are being proxied. In contrast, explicit proxies are those which a client is aware of and/or to which a client is directed (e.g., by configurations to a browser) to make requests.
Like firewalls, proxies can filter traffic based on many packet attributes, such as source IP address and/or port, and destination IP address and/or port. In addition, proxies can filter traffic based on destination service such as hypertext transfer protocol (HTTP), file transfer protocol (FTP), etc., and on other attributes. As these devices operate at the application layer, they may inspect the contents of the traffic, blocking what a network administrator views as inappropriate content. A proxy can route traffic at the network layer if the traffic satisfies the proxy's policies. In addition, a proxy can terminate all connections, and may initiate its own connections if the connections satisfy the proxy's policies.
One benefit of using a proxy to perform filtering operations is that the proxy can “understand” certain applications and protocols (such as HTTP, FTP, etc.), and can detect whether an unwanted protocol is being transmitted on a non-standard port, or whether a protocol is being abused in a known harmful way. When performed by a second-generation firewall, this type of filtering is usually done by deep packet inspection (DPI). DPI is a form of packet filtering that examines the data part of a packet, searching for non-protocol compliance or predefined criteria (i.e., data patterns) to decide if the packet can pass. This is in contrast to shallow packet inspection, performed by first-generation firewalls, which just checks the header portion of a packet. DPI devices have the ability to look at OSI Layer 2 through Layer 7 information within a packet and identify and classify traffic based on packet signatures (i.e., known packet characteristics associated with malicious or undesired content).
Devices operating in the OSI Layer 2-Layer 4 (L2-L4) protocol stack layers offer the advantages of simplicity and high speed, but are relatively unintelligent, and are able to base the regulation of IP packets only on information available in the link-layer header, IP packet header and L-4 protocol headers (e.g., such as TCP and UDP headers). Such devices can block traffic from undesirable IP addresses, for example, but are not capable of detecting policy violations for content transmitted from a “trusted” address. Devices operating at OSI Layer 7 (L7), on the other hand, are capable of regulating IP packets based on the payload of a single packet or of a plurality of individual packets making up a particular message. Such devices provide network administrators with much more visibility into and control of network communications. However, this visibility and control comes at an increased overhead cost reflected in more expensive processing hardware and longer processing times. Given the ever expanding flow of IP-based communication traffic, this overhead is a substantial and worsening problem for many private networks. Moreover, conventional IP firewall methods of regulating traffic require the rigid application of “rules” which must be applied in a particular and inflexible order. Often, these rules are not readily amenable to modification without jeopardizing the integrity of the policies as the needs of the enterprise and the nature of the threat change.